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Summary:  Military  systems  at  all  levels  of 
decision  making  require  the  ability  for  the  decision 
maker  to  find  information,  query  it,  seek  refinement 
of  it,  process  it  in  combination  with  other 
information,  add  value  to  it,  make  the  decision  and 
communicate  the  decision  to  another  user.  The  US 
Office  of  Naval  Research  program  in  Interactive 
Multimedia  and  User-Centered  Design  supports 
research  in  how  to  employ  technological 
capabilities  to  enhance  a person’s  abilities  to  carry' 
out  the  decision  making  objective.  This  paper  will 
employ  several  research  projects  from  this  program 
to  illustrate  basic  findings  that  impact  how  to 
design  systems  to  meet  this  objective.  Usability 
objectives  require  that  design  address  the  impact  on 
the  user’s  ability  to  perform  the  task.  The  research 
reported  here,  even  though  in  domains  or 
applications  that  differ  from  battle  management  are 
at  the  level  of  studies  of  enabling  understanding  of 
the  design  space.  Reported  results  provide 
guidance  and  suggest  how  to  design  the  information 
presentation  in  the  appropriate  form  for  its  use. 

First,  basic  assumptions  behind  development  of  the 
program  will  be  described  to  set  the  context  for  the 
projects.  Then,  each  project  will  be  summarized 
presenting  the  research  findings.  The  selected 
projects  provide  scientific  bases  for  design 
decisions  that  impact  how  a person  can  actually  use 
information  in  different  presentation  contexts, 
multimedia  documents,  multidimensional  flat  panel 
displays  and  in  a Responsive  Workbench  context. 
Other  research  includes  multiple  modalities  but  will 
not  be  discussed  here. 

Background:  The  program  in  Interactive 

Multimedia  and  User-Centered  Design  assumes  an 
integrated  multidisciplinary  approach  to  employing 
cognitive  modeling  and  perceptual  understanding  in 
design  of  task-focused  interactive  systems.  Teams 
of  researchers  include  cognitive  scientists, 


psychologists,  computer  scientists,  and  experts  in 
experimental  design  in  differing  combinations. 
Research  projects  are  theory  and  hypothesis  based 
allowing  effective  evaluation  to  produce  better 
understanding  of  the  human  component  of  a system 
and  hence,  contributing  to  effective  system 
usability  and  design.  Computational  cognitive 
models  such  as  ACT-R  (Anderson,  1993  ) , 
COGNET  ( Zachery,  Ryder  and  Hicinbothom, 
1999)  and  EPIC/GLEAN  (Meyer  and  Kieras, 
1997a,  1997b)  are  being  developed  to  provide 
resources  for  cognitive  appraisal  of  design 
decisions. 

In  addition,  building  usable  systems  entails  analysis 
of  which  modalities  are  best  for  what  types  of 
information  or  information  use,  such  as  vision, 
audition,  or  somatosensory,  and  what  media  can 
provide  them,  such  as  a monitor,  film,  3-D  sound, 
etc.  The  space  is  further  complicated  in  that 
information  presentation  can  have  multiple  modes 
that  also  differ  in  how  a human  can  deal  with  them, 
such  as  in  the  visual  modality  on  a monitor,  one  can 
have  graphs  and  text,  or  text  and  an  animated 
drawing  or  only  text  as  modes.  These  three 
dimensions,  modality,  media  and  mode  are  being 
investigated  for  better  understanding  of  their 
impact  on  the  user’s  ability  to  deal  with  the 
information  being  presented. 

With  this  brief  introduction  of  some  of  the  features 
of  the  design  space  that  must  be  addressed  to  obtain 
effective  design,  the  remainder  of  the  paper  will 
illustrate  how  some  of  them  effect  usability  of 
information  by  presenting  research  investigating: 
Interactive  multimedia  document  design;  Visual 
presentation  of  information  using  perspective  view 
technology,  2-D  or  3-D  on  flat  panels;  and 
Usability  in  Virtual  Environments  (VE),  user- 
focused  information  management  in  an  immersive 
Responsive  Workbench  environment. 


Paper  presented  at  the  RTO  HFM  Symposium  on  "Usability  of  Information  in  Battle  Management 
Operations  ”,  held  in  Oslo,  Norway,  10-13  April  2000,  and  published  in  RTO  MP-57. 
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Interactive  Multimedia  Document  Design:  Dr. 

Mary  Hegarty  (University  of  California  Santa 
Barbara)  and  Dr.  N.  Hari  Narayanan  (Auburn 
University)  have  teamed  to  investigate  a cognitive 
theoretical  model-based  multimedia  design  for 
manuals  that  are  intended  to  teach  how  devices 
work.  Such  manuals  are  a focus  of  many  new 
requirements  that  insist  on  on-line  information  for 
training  and  reference.  They  are  developing 
methods  for  presenting  information  to  maximize 
the  user’s  ability  to  understand  causal  relationships 
from  it.  Their  theory  includes  a model  of  human 
states  of  comprehension  of  mechanical  devices  (the 
domain  of  previous  work)  and  their  operations. 
The  theory  includes  a four  step  process  in  obtaining 
understanding  of  how  a device  works  to  be  able  to 
apply  the  understanding  to  a real  world  instance  of 
the  device.  The  theory  of  comprehension  includes 
the  following  stages:  Stagel:  Machine 

Decomposition  by  diagram  parsing;  Stage  2: 
Constructing  a Mental  Model-  Making 
Representational  Connections;  Stage  3: 
Constructing  a Static  Mental  Model—  Making 
Referential  Connections;  Stage  4:  Determining  the 
Causal  Chain  of  Events;  and  Stage  5: 
Constructing  a Dynamic  Mental  Model  by 
Mental  Simulation  and  Rule-based  Inference 
(Narayanan  and  Hegarty,  1998). 

Their  method  for  design  recognizes  cognitive 
workload  factors  and  attempts  to  reduce  them 
during  reading  and  studying  of  the  material.  They 
are  studying  how  the  integrated  placement  of  text 
explanations  and  diagrams,  both  static  and 
animated,  contributes  to  the  understanding  of 
concepts  and  processes  in  a hyperlinked  document. 
The  model  is  based  on  previous  work  on  relative 
location  of  text  and  graphics,  showing  that  text 
associated  with  an  image  must  be  nearby  and  that 
labeling  in  the  text  must  be  clearly  visible  on  the 
drawing  (Hegarty,  1992;  Mayer,  1989).  They 
designed  on-line  documents,  based  on  the  theory  of 
stages  just  described,  in  the  context  of 
understanding  how  a flushing  cistern  (toilet)  works. 
Understanding  requires  inferring  causal 
relationships  after  the  components  are  recognized 
and  integrating  them  with  world  knowledge  about 
water  flow  and  how  the  mechanical  components 
work  in  isolation. 

The  initial  design  of  the  hypermedia  manual 
integrating  this  theory  produced  a complex  set  of 


experiments  to  validate  the  theory.  The  studies 
compared  learning  and  understanding  using  a 
hyperlinked  version  of  the  information  as  well  as  a 
static  paper  version,  both  were  created  using  the 
theory  of  comprehension  summarized  above.  The 
results  indicate  that  presenting  information  in  a 
hypermedia  manual  that  includes  hyperlinks, 
colored  diagrams,  animations  (rather  than  static 
diagrams)  and  commentaries  (rather  than  visual 
text)  had  no  effects  on  comprehension.  The  same 
results  were  obtained  by  expository  causal  link 
information  in  the  text  and  in  some  cases  just  a 
labeled  diagram  alone  conveyed  the  same  level  of 
understanding.  It  seems  that  the  juxtaposition  of 
the  structure  and  content  of  closely  associated 
material  is  more  important  than  the  form  of  the 
document(  Hegarty,  Quilici,  Narayanan,  Holmquist 
and  Moreno,  1999). 

Among  their  findings  is  the  fact  that  they  were 
unable  to  adequately  evaluate  their  theory  and 
model  of  design  and  its  predictions  in  the  context  of 
the  document.  Analysis  of  their  results  have 
suggested  other  ways  to  implement  their  principled 
theory  in  the  context  of  interactive  multimedia 
information  presentations.  Current  work  continues 
to  find  that  the  “message  is  more  important  than  the 
medium”  as  far  as  using  technology  to  enhance 
understanding  or  comprehension. 

Additionally,  Hegarty  and  Narayanan  found,  as  has 
been  documented  elsewhere,  that  animation  does 
not  necessarily  aid  understanding  and  that  it  can 
confuse  the  user.  Several  results  indicate  that 
animation  must  be  well  integrated  with  text,  or 
needs  to  be  integrated  into  material  in  a manner  that 
requires  interaction  with  the  concepts  in  addition  to 
exploration  of  the  animated  rendering  of  it  (Hegarty 
and  Sims,  1994;  Kehoe,  Stasko,  and  Taylor,  1999). 

They  are  continuing  their  investigation  on 
understanding  multimedia  information  using 
atmospheric  weather  as  their  focus.  It  is  something, 
not  directly  observable  but  often  visualized  for 
understanding,  such  as  weather  front  movements 
and  wind  speeds  and  directions.  Users  will  be 
evaluated  for  their  retention  of  information  and 
understanding  of  causal  relationships  somewhat 
similar  to  the  original  study  of  how  a pumping 
cistern  works. 

Others  have  been  studying  the  effectiveness  of 
animation  in  learning  contexts.  The  fact  that  its 
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role  is  controversial  even  though  assumed  to 
enhance  learning  has  implications  for  the  use  of 
animation  in  battle  management  contexts.  Dr.  John 
Stasko  and  his  Information  Visualization  Group 
(Georgia  Institute  of  Technology)  has  shown  that 
animation  improves  understanding  in  certain 
contexts  and  has  been  exploring  how  to  best 
provide  animation  capabilities  in  learning  systems 
(Stasko,  1996). 

Empirical  evidence  suggests  that  application  of 
animation  in  learning  and  possibly  in  understanding 
information  in  decision  making  environments  may 
depend  on  knowledge  of  the  context  of  application, 
the  mental  workload  of  the  person,  and  whether  the 
person  has  the  ability  to  mentally  visualize  and 
dynamically  manipulate  it.  In  understanding 
algorithms,  or  software  processes,  when  the  person 
already  understands  a principle  and  is  able  to 
somehow  visualize  it,  the  animation  seems  to 
provide  enhanced  learning  (Kehoe,  Stasko,  and 
Taylor,  1999).  The  animation  provides  additional 
depth  of  understanding.  This  was  demonstrated  in 
the  context  of  animations  to  explain  how  software 
algorithms  worked,  a phenomena  that  has  no  real 
world  visualization.  It  seems  that  animation  is  only 
useful  as  an  enhancement  to  understanding  when 


the  person  is  already  able  to  mentally  visualize  the 
action  or  concept. 

Visual  Presentation  of  Information  Using 
Perspective  View  Technology:  Drs.  Michael 

Cowen  (SPAWAR  Systems  Center-San  Diego)  and 
Mark  St.  John  (Pacific  Science  and  Engineering 
Group,  Inc.)  are  investigating  the  role  display 
formats  play  in  the  ability  of  a user  to  access  and 
use  information.  The  issue  can  be  stated  as,  what 
type  of  information  is  best  presented  in  what 
format?  In  a recent  technical  report  (St.  John  and 
Cowen,  1999)  results  from  experiments  comparing 
two  presentation  formats  to  determine  answers  to 
the  question  about  form  and  available  content  have 
shown  that  in  tasks  where  relative  position  is 
important,  the  multiple  projection  2-D  presentation 
provides  explicit  information  that  can  easily  be 
determined  with  little  error.  In  contrast,  the  3-D 
presentation  has  increased  ambiguity  due  to  the 
perspective  rendering  of  the  same  information.  See 
Figure  1.  If  the  task  is  to  determine  the  shape  or 
general  configuration  of  an  object,  then  the  3-D  is 
superior  to  the  2-D  multiple  view  perspective  (St. 
John  and  Cowen,  1999). 


I 

I 
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Based  on  these  results,  subsequent  investigations 
were  carried  out  to  determine  the  implications  of 
these  findings  in  an  Area  Air  Defense  Display.  The 
system  design  included  several  3-D  presentations, 
of  terrain  and  of  icons,  and  the  goal  was  to 


determine  whether,  the  3-D  perspectives  and 
realism  enhanced  situation  awareness  or  merely 
provided  a more  satisfying  feel  to  the  picture. 
Examples  of  the  icons  used  in  the  experiment  are 
provided  in  Figure  2. 


Figure  2:  Examples  of  icon  conditions  for  experiment,  left  to  right.  East  heading  bomber  depicted 
realistically  in  3-D,  realistically  in  2-D,  and  symbolically  or  non-realistically  (Smallman,  Schiller,  and 

Mitchell,  1999). 


The  icons  were  overlaid  on  a flat  panel  3-D  terrain 
map  in  perspective  view'  in  one  condition,  and  in 
planar  view  in  the  other  two  conditions.  Tasks 
include  track  identification  and  monitoring  to  gain 
situation  awareness.  Relative  ground  position  was 
provided  by  the  corresponding  shadow  on  the 
terrain  surface  in  the  perspective  view  condition. 
The  scenario  used  was  the  same  for  all  three 


conditions,  1)  3-D  realistic  icons  on  a 3-D 
perspective  terrain  presentation,  2)  3-D  realistic 
icons  on  a planar  terrain  presentation  ( no  shadows 
on  surface  and  bird’s  eye  view)  and  3)  2-D  non- 
realistic  icons  on  a planar  terrain  presentation. 
Figure  3 shows  the  2-D  iconic  representation  on  a 
planar  terrain  presentation  as  an  example,  condition 
3 just  discussed. 


Figure  3:  2-D  non-realistic  icon  view  on  a planar  view  map  (Smallman,  Schiller,  and  Mitchell,  1999). 


Results  showed  that  the  2-D  iconic  representations 
are  the  easiest  to  remember  and  produce  the  fewest 
errors  on  identification  and  reporting  of  the  tested 
attributes.  While  the  3-D  icon  on  the  3-D 


perspective  terrain  presentation  was  assumed  by  the 
design  to  provide  the  best  depiction  to  facilitate  the 
user  task,  the  results  showed  that  the  opposite 
holds.  Due  to  the  foreshortening  of  the 
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perspectives  and  the  nature  of  the  shadow 
connectivity  with  the  icon,  ambiguities  are 
introduced  that  produce  errors  in  the  ability  of  the 
user  to  track  the  attributes  of  the  object  represented 
by  the  icon.  In  addition,  when  the  icons  are  small 
due  to  the  distance  above  the  ground,  it  is  very 
difficult  to  find  the  corresponding  shadow  for  the 
aircraft.  Decluttering  the  image  may  provide  some 
help,  but  adds  another  subtask  for  the  user. 
Furthermore,  in  the  3-D  case,  the  light  source  and 
its  effect  on  how  the  shadows  appear  introduces 
another  aspect  of  ambiguity  in  the  presentation. 
Details  of  these  investigations  and  full  results  can 
be  found  in  Smallman,  Schiller,  Mitchell,  1999. 

Battle  management  systems  require  many  tasks  and 
manipulations  of  information  similar  to  those  that 
were  used  in  these  studies.  The  usability  issues 
scientifically  documented  in  these  studies  are 
important  and  generally  ignored  in  current  designs. 
Understanding  limitations  on  visual  presentation 
options  and  how  they  interact  with  user  tasks  and 
with  user  capabilities  needs  to  be  further  explored. 

Usability  in  Virtual  Environments:  On-going 

joint  research  by  Dr.  Deborah  Hix  at  Virginia 
Polytechnic  Institute  and  State  University  ( 
Virginia  Tech)  and  by  Dr.  Edward  Swan  II  at  the 
Naval  Research  Laboratory  (NRL)  focuses  on 
methods  for  defining  usability  criteria  in  virtual 
environment  (VE)  design.  Their  recent  award 
winning  paper  at  the  Virtual  Reality  99 
Conference,  (Hix,  Swan,  Gabbard,  McGee,  Durbin 
and  King,  1999)  exemplifies  a four  step  method  of 
usability  design  that  was  verified  while  applying  it 
in  a Naval  Battlespace  context.  This  section  will 
begin  with  an  overview  of  a preliminary 
investigation  into  the  state  of  usability  for  VE  and 
will  conclude  with  an  overview  of  the  methodology 
and  its  application  in  the  battle  management 
domain. 

Initial  questions  regarding  usability  issues  in  VE 
arose  during  discussions  about  device  selection  and 
use  and  the  fact  that  the  metaphors  for  use  in  VEs 
are  far  more  complex  and  allow  many  new 
interaction  possibilities  than  any  other  technology 
to  date.  Additional  evidence  that  human  usability 
issues  need  attention  has  been  documented  in  the 
findings  that  humans  subjected  to  VEs  suffer 
performance  effects  even  up  to  an  hour  after  they 


leave  the  immersion  (Kennedy  and  Stanney,  1996). 
Previous  enhancements  to  VE  technology  that 
focused  on  presentations,  visual,  auditory,  haptic, 
and  their  refinement  was  able  to  only  ameliorate 
part  of  the  effect  on  the  user.  Since  the  VE  provides 
an  immersive  interactive  space  for  design  and  use, 
it  presents  a design  challenge  along  a vast  number 
of  dimensions. 

An  initial  effort  to  document  a taxonomy  of 
usability  characteristics  for  VEs  intended  to  begin  a 
dialogue  that  would  aid  in  understanding  the  design 
space  at  a theoretical  level  and  that  would  aid  in 
effective  VE  design.  Dr.  Hix  with  her  then 
graduate  student,  Joseph  Gabbard,  began  their 
effort  to  understand  the  usability  issues  regarding 
design  of  VEs.  They  did  a thorough  review  of 
relevant  literature,  interviewed  leading  VE 
researchers  and  designers,  and  talked  with 
practitioners  and  users  of  VE  systems,  chiefly  in 
training  or  mission  rehearsal  domains.  They  found 
that  the  usability  space  for  VEs  included  complex 
interdependencies,  among  users,  user  tasks,  input 
devices,  output  devices,  etc.  The  result  of  their 
investigation  is  a taxonomy  of  usability  methods 
(Gabbard  and  Hix,  1997). 

Employing  known  usability  methods  as  a start 
point,  they  found  that  VE  expanded  their 
applicability  to  teams  and  integrated  team  tasks,  as 
well  as  tasks  that  involved  the  whole  human  as 
opposed  to  using  a system  while  sitting  in  front  of  a 
monitor.  Immersion  as  part  of  usability  requires 
new  metaphors  in  thinking  about  what  usability 
means.  The  framework  in  the  document  organizes 
user  interaction  design  guidelines  and  discussion 
into  four  major  areas:  1)  users  and  user  tasks,  2) 
input  mechanisms,  3)  virtual  models,  and  4) 
presentation  mechanisms.  All  findings  are  cross- 
linked  and  documented  with  interview  explanations 
or  cross-referenced  to  other  related  usability.  The 
document  intended  as  a dynamic  taxonomy, 
provides  a broad  definition  of  the  state  of  usability 
in  VEs.  It  is  available  as  a public  resource  at  URL: 
http:/ /csgrad . cs.  vt.  edu/~i  gabbard/ve/taxonomv.  Figure 
4 provides  an  overview  of  the  taxonomy  areas. 
This  characterizes  the  complex  design  space  and 
illustrates  how  usability  can  impact  the  design  of 
VEs. 
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Figure  4:  An  Overview  of  the  Taxonomy  (Gabbard  and  Hix,  1997). 


Follow-on  joint  work  at  NRL  integrated  many  of 
the  basic  findings  into  a defined  method  for 
usability  evaluation  in  the  context  of  battle 
information  management.  This  work  employed  an 
immersive  environment  on  an  Responsive 
Workbench.  In  addition  to  developing  a suitable 
interaction  device,  the  effort  took  approximately 
nine  months  to  apply  the  usability  method, 
validating  it  as  the  work  progressed  and  in  the  end, 
producing  a system  that  is  ready  for  summative 
evaluation  in  the  application  domain. 

This  development  has  supported  the  fact  that  by 
carrying  out  the  up-front  usability  design  methods, 
life-cycle  cost  reduction  can  be  realized.  A few  of 
the  many  discoveries  in  the  context  of  this  work 
will  be  summarized.  Below,  Figure  5,  is  the 
workbench  environment,  Dragon  (Durbin  et  al., 
1998),  including  the  terrain  and  object  renderings 
that  form  the  basis  of  the  design  solution.  It  is  the 


result  of  a successful  application  of  the  design,  test, 
design  cycle  of  usability  assessment.  Details  of  the 
effort  can  be  found  in  the  Virtual  Reality  99  paper 
referenced  above.  The  methodology  is  elaborated 
in  Gabbard,  Hix  and  Swan,  1999.  The  usability 
method  includes  a four  step  assessment,  based  on 
the  design  space  presented  in  Figure  4.  The  user- 
centered  methodology  assumes  sequential 
performance  of  the  following  4 steps:  1)  user  task 
analysis,  2)  expert  guidelines-based  evaluation,  3) 
formative  user-centered  evaluation,  and  4) 
summative  evaluation. 

Cyclic  evaluations  are  performed  as  design 
problems  arise.  As  more  problems  are  eliminated 
in  the  early  phases,  the  cost  of  implementation  and 
subsequent  possible  modification  are  reduced.  The 
four  steps  will  be  briefly  described  In  the  context  of 
developing  the  interface  design  for  navigating 
within  the  Dragon  workbench  environment. 
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Figure  5:  Dragon  Responsive  Workbench  Design  (Hix  et  al.,  1999) 


1. 

1.  User  task  analysis:  must  include  an 

understanding  that  is  as  comprehensive  as 
possible  of  the  tasks  the  user  will  be  doing  and 
the  goals  of  the  total  system  use  to  achieve 
good  user-centered  design.  It  was  found  very 
early  in  working  with  the  workbench,  that  the 
ability  of  the  user  to  move  around,  locomote,  to 
find  the  information  they  need  is  critical  to  any 
other  tasks.  Therefore,  the  task  selected  to 
implement  and  evaluate  the  method  became 
locomotion. 

2.  Expert  guidelines-based  evaluation: 

identifies  potential  usability  problems  by 
evaluating  a user  interaction  design  against 
what  is  known  by  experts  to  be  sound  design 
principles.  Any  violations  are  revisited  and  the 
design  is  redone  to  address  them.  Multiple 
experts  perform  the  evaluation,  independently 
and  then  in  joint  discussion.  During  this  phase 
in  the  design  of  Dragon,  experts  discovered  a 
major  problem  of  poor  mapping  of  locomotion 


tasks  (pan,  zoom,  pitch,  heading)  to  flight  stick 
buttons  as  well  as  problems  with  inadequate 
graphical  and  textual  feedback  to  the  user  about 
the  locomotion. 

3.  Formative  user-centered  evaluation:  requires 
careful  definition  of  scenarios  that  will  exercise 
the  interface  under  the  set  of  tasks  included  in 
the  design.  In  the  Dragon  design  of 
locomotion,  the  initial  control  was  to  allow  the 
user  to  move  the  map  relative  to  themselves. 
During  formative  evaluation,  users  indicated 
they  wanted  to  be  able  to  fly  over  the  terrain. 
This  input  suggested  a different  design  that  was 
developed  to  allow  movement  in  exocentric 
(the  map  movement)  and  egocentric  ways(  the 
fly  -over).  Subsequent  evaluations  were 
carried  out  to  build  the  right  responsiveness 
into  the  interface  to  adequately  capture  the 
control  from  the  user’s  perspective  for  both 
these  metaphors.  This  evaluation  includes  data 
collection  about  user’s  performance  that  guides 
the  design  for  the  system. 
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4.  Summative  evaluation:  requires  comparative 
evaluation  between  several  competing  designs. 
This  is  a formal  evaluation  of  the  implemented 
user-focused  interface.  The  affects  of 
exocentric  vs  egocentric  control  are  one 
variation  being  studied  in  Dragon.  To 
effectively  test  this,  all  actions  must  be 
accomplishable  in  both  systems  to  be  able  to 
compare  measures  that  will  determine  the  best 
design.  Speed  of  problem  solving  is  one  such 
measure  as  is  correctness  of  solution.  For  this 
type  of  evaluation  an  adequate  scenario  test  set 
must  be  defined  that  drives  the  interface  use  in 
ways  that  will  stress  its  functionality  to  firmly 
determine  the  best  solution.  Summative 
evaluation  focusing  on  egocentric  versus 
exocentric  control  is  currently  underway 
(Gabbard,  Hix  and  Swan,  1999.) 

The  importance  of  this  applied  methodology  is  that 
the  right  level  of  detailed  evaluation  is  performed  at 
the  right  time  during  the  design,  saving  extensive 
redesign  to  correct  problems  late  in  the  cycle.  The 
possibility  of  achieving  an  adequate  and  effective 
system  is  greatly  enhanced  and  at  a much  reduced 
overall  cost. 

Conclusions:  Usability  methods  have  many 

applications  in  military  systems.  We  are  just 
beginning  to  understand  some  of  the  underlying 
cognitive  aspects  that  are  reflected  in  how  people 
can  obtain  and  use  information.  As  more 
interdisciplinary  conversations  occur  between 
cognitive  scientists,  usability  engineers,  human- 
computer  interface  designers  and  domain  experts 
we  will  see  a positive  change  in  the  design  of 
systems — more  usable  by  more  people  without 
restrictive  selection  criteria  and  the  overall  result 
will  be  a cost  reduction  in  the  life-cycle  costs  of  the 
systems. 
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